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ABSTRACT 

While technical training and advanced degree’s assure 
proficiency at specific tasks within engineering 
disciplines, they fail to address the potential for 
communication breakdown and decision making errors 
familiar to multicultural environments where language 
barriers, intimidating personalities and interdisciplinary 
misconceptions exist. In an effort to minimize these 
pitfalls to effective panel review, NASA’s lead safety 
engineers to the ISS Safety Review Panel (SRP), and 
Payload Safety Review Panel (PSRP) initiated training 
with their engineers, in conjunction with the panel 
chairs, and began a Panel Resource Management (PRM) 
program. The intent of this program focuses on the 
ability to reduce the barriers inhibiting effective 
participation from all panel attendees by bolstering 
participants’ confidence levels through increased 
communication skills, situational awareness, debriefing, 
and a better technical understanding of requirements 
and systems. 

1. BACKGROUND 

In the late 1970’s, research performed by NASA and the 
Federal Aviation Administration (FAA) found a number 
of preventable aviation mishaps resulting from the 
improper, ineffective, or nonexistent use of available 
cockpit/crew resources, resulting in the study, creation 
and their subsequent implementation of a new training 
tool: Cockpit/Crew Resource Management, or CRM. 1 1 

The scenarios surrounding the applicability of CRM 
initially centered on the dynamic, high stress, cockpit 
environment, but eventually spread into other 
occupational fields to include medical emergency and 
operation rooms, and fire stations. Though the realm of 
system safety reviews rarely reaches the high intensity 
of flight emergencies, or medical procedures, the need 
to maintain an effective and involved system safety 
review often becomes hampered by many of the same 
breakdowns and barriers common within these group 
dynamics: Communication breakdowns, lack of 

situational awareness, poor or ineffective decision 
making, and teamwork. 


1 Cooper, White, & Lauber. Resource Management on 
the Flightdeck, Ames Research Center; 1980. 


To that aim, NASA’s contracted Safety and Mission 
Assurance (S&MA) lead safety engineers supporting 
both the ISS Safety Review Panel (SRP), and Payload 
Safety Review Panel (PSRP), set out to confront these 
barriers to effective, multilateral discussions within a 
panel, altered the CRM training and implementation 
from a focus on high intensity environments, and 
created a tool better tailored for safety review panel 
settings and refer to this new tool as Panel Resource 
Management, or PRM. 

As with any new program, the largest hurdle to 
overcome resides with that program’s implementation 
and the subsequent evaluation of that program’s 
effectiveness. To surmount this challenge, we utilize a 
phased PRM implementation approach adhering to the 
following six steps: 

• Assessing the status of the panel before 

implementation, 

• Receiving commitment from all managers (starting 
with senior managers), 

• Customizing the training to reflect the nature and 
needs of the organization, 

• Defining the scope of the program and 

implementation plan, 

• Communicating the nature and scope of the 

program before startup, 

• Instituting quality control procedures 

In the process of implementing PRM, not only did we 
mold our common barriers training to familiar scenarios 
often manifested with safety reviews, but phased in a 
“back to basics” training for a concentrated study and 
application of core safety regulations and ISS systems. 

1.1 NASA Safety Panel Structure 

To best understand PRM implementation requires an 
understanding of basic panel structure. Multiple safety 
panels exist within NASA’s Johnson Space Center 
(JSC), but for the purpose of this paper, we address the 
International Space Station Safety Review Panel (SRP), 
and the Payload Safety Review Panel (PSRP). Each of 
these panels are governed by their charters, the PSRP 



with JPC1 152.4M - SSP/ISS Program Payload Safety 
Review Panel, and similarly so, the SRP. 

Though different panels, they basically share identical 
responsibilities; the responsibility for conducting safety 
reviews in accordance with National Space 
Transportation System (NSTS)/International Space 
Station 13830, "Payload Safety Review and Data 
Submittal Requirements," and to assure the 
implementation of either SSP 50021 or NSTS 1700.7, 
"Safety Policy and Requirements for Payloads Using the 
Space Transportation System;" the "International Space 
Station Addendum;" and NSTS/International Space 
Station 18798, "Interpretations of NSTS/International 
Space Station Payload Safety Requirements." 

The Panel assists hardware provider and payload 
organizations in assuring that safety critical payload 
subsystems are appropriately verified. Specifically, 
NASA tasks the Panels with the following functions: 

• Assists payload organizations in the interpretation 
of safety requirements consistent with referenced 
NASA issuance, for each payload and provides 
recommendations for implementation. 

• Conducts safety reviews as appropriate during 
various phases of payload development. 

• Evaluates modifications to payloads that either 
affect a safety critical subsystem or create a 
potential hazard to the Orbiter, Space Station, or 
crew. 

• Evaluates the safety analyses, safety reports, and 
waiver/deviation requests prepared by payload 
organizations. 

• Assures the resolution of payload safety issues. 

Due to similarity in nature, the basic panel construct for 
the purposes of this paper consists of: 

• The Panel Chair, who maintains responsibility for 
the final disposition of hazard reports and non- 
compliance reports (NCRs) 

• Safety and Mission Assurance Office (Safety 
Engineer function, who tracks assigned flight 
hardware, payloads, and operational safety 
assessments) 

• Flight Crew Operations Directorate, representing 
the Astronaut corps 

• Mission Operations Directorate 

• Engineering Directorate 

• Space and Life Sciences Directorate 

• Space Shuttle Systems Integration Office 

• Kennedy Space Center - Safety and Quality 
Assurance Directorate 

• International Space Station Safety and Mission 
Assurance Office (for International Space Station 
Payloads) 


• Extravehicular Activity (EVA) Project Office (for 
Extravehicular Activity Payloads) 

• Safety and Mission Assurance Office (Executive 
Secretary Position)International Space Station 
Payloads Office (for International Space Station 
Payloads) 

Each panel member plays an integral role bringing 
specific expertise in their designated arena essential for 
successful disposition an review of all hazard 
assessments brought before the panel. The remainder of 
this paper discusses methods to enhance the roles and 
responsibilities of each of these members for the benefit 
of the panel chairman, and flight safety as a whole. 

2. UNDERSTANDING CRM 

What is Crew Resource Management? Simply put, 
CRM is the effective use of all resources. Broken down 
into specifics, CRM promotes the effective use of 
resources by improving upon the following key factors 
usually responsible for breaking down a team’s 
cohesiveness. 

2.1 Communication 

Communication is the key to success in any endeavor. 
Misunderstandings lead to errors and mistakes. CRM 
teaches people to focus on the communication model 
(sender-message-medium-receiver- feedback), speaking 
directly and respectfully and communicating 
responsibility. 

2.2 Situational Awareness 

Situational awareness conceptually discusses the need to 
maintain attentiveness during an event. It discusses the 
effects of perception, observation and stress on 
personnel. There is emphasis on the need to recognize 
that situations in the space systems reviews are 
particularly complex and require full attention. 

2.3 Teamwork 

Any group that fails to perform as a team is eventually 
doomed to fail. Failure in the safety review panel 
settings results in poor panel reviews potentially 
yielding the unthinkable loss of personnel and major 
flight hardware. CRM training emphasizes team 
performance through exercises in the awareness tier and 
panel interactions during the reinforcement tier. The 
training also focuses on “leadership-followership” so all 
members understand their place on the team and the 
need for mutual respect. 



2.4 Decision Making 

Decision making describes choices made based on the 
information at hand. Safety panel decision making 
relies heavily on risk/benefit analysis. Too little 
information results in poor risk assessment by decision 
makers and results in errors, and/or the acceptance of 
inadequate designs. Too much information overloads 
the panels and makes it difficult to make effective 
decisions. PRM training concentrates on giving and 
receiving information so safety review panels make 
appropriate decisions. 

2.5 Debriefing 

During any given flight, a great majority of “learning” 
or training occurs after the event during the debriefing. 
The briefer, typically the assigned safety engineer, or in 
flight terms, the formation commander and/or aircraft 
commander, structures the debrief to address any and all 
significant events (both positive and negative) that 
occurred during the safety review/mission for the 
benefit of all participants to understand and agree upon 
what occurred and recognize potential improvements. 

3. CRM BARRIERS APPLIED WITHIN SAFETY 
PANELS 

3.1 Barriers to Communication 

The setting of a flight safety review, with a multi- 
member panel and hardware providers, is ripe for the 
opportunity of miscommunication or barriers that hinder 
the communication process. As with any conversation 
or negotiation, challenges arise from the set up of the 
discussion to negotiating an update for a hazard report. 
The panel with its diverse membership can vary in its 
direction of approach to a problem and work towards a 
solution. This can result in the panel being split with 
multiple discussions and decisions/assumptions being 
made. When this is combined with a hardware provider 
presenting new and difficult design modification, one 
may wonder how any conclusion or resolution is ever 
reached. 

As the atmosphere and partnership in space business 
tend toward international relations, it is inevitable that 
cultural and language barriers can and do arise in a 
safety review. However, language barriers can also 
occur when communicating with people from your own 
country. Each Payload Organization or Hardware 
Providers that comes before a safety panel brings with 
him his own set of experiences and preferences that can 
challenge the communication process. 

Barriers also come in the form of the logistics of the 
conference room. For example, a panel member that has 


trouble deciphering some verbal noises choosing to sit 
in the back of the room, far away from the speaker. 

Typically though, the major hindrance to the safety 
review process is the interaction between the presenters 
of the hardware information and the panel members. All 
too often reviews turn into a scenario such as: presenter 
talks, panel talks, presenter talks, panel talks. What is 
lost in this cycle is the basic principle of listening to 
what the presenter is saying. Without listening, there is 
no way to adequately understand what is being said 
before responding. 

3.2 Situational Awareness Breakdowns 

If the need to maintain attentiveness during an event 
defines situational awareness, then any derailment from 
one’s attention span creates a situational awareness 
breakdown. In order to establish situational awareness, 
human beings take in information through the 5 senses - 
touch, hearing, smell, sight and taste - and also sub- 
consciously or intuitively. This information is then 
transformed by the brain into a mental model of the 
situation, a process known as perception. The perceptive 
process depends not merely on current information for 
its evaluation of the situation but also takes account of 
past experience. Perception is therefore a product not 
only of immediate sensations but also of cultural and 
social influences acquired through a life-time of 
experiences. Accordingly, because of the different 
factors which have shaped their lives, individuals 
interpret situations differently. These same factors can 
negatively influence people through a loss of situational 
awareness by providing false information, quite simply 
by subjecting themselves to a number of degrading 
influences such as inattention, distraction, under- 
arousal, stress, boredom, fatigue, etc, etc. 

Try though we might, sometimes panel meetings just do 
not hold everyone’s attention all the time. Some 
meetings breeze through hazard reports to the tune of 4 
an hour (generally for more benign causes such as 
materials compatibility, some structures and 
mechanisms reports, and simple electrical mating and 
de-mating), while the panel may spend 3 days on a 
single hazard report and 4 hours on one cause within a 
hazard report. Examples of these more complex reports 
include Guidance, Navigation and Control (GNC) with 
respect to collision with visiting vehicles and the 
International Space Station (ISS), or ground control and 
monitoring of robotic systems. 

The trap panel members’ face under these 
circumstances is easy to spot if people know the causes 
for the loss of situational awareness. If a safety panel 
addresses multiple reports on multiple subjects, then 



many different panel members are engaged at any given 
point in time. However, it becomes quiet easy for the 
human mind to wander over a prolonged period of 
disengagement. 

Take the following example. During a portion of the 
ISS assembly sequence EVA crewmembers need to 
install an overboard Hydrogen Vent Valve in place of 
an existing water vent valve. For more than an hour, the 
conversation centered on the structural fatigue of the 
existing water vent valve and the required torque needed 
to replace it. Conversation then turned to verifying the 
number of existing seals on the pressure wall feed- 
through along with the intra-vehicular activity (IVA) 
tool and operations required for that procedure. Clearly, 
this conversation centered on MOD operations and 
engineering directorate opinion. However, the entirety 
of this operation centered on the amount of force an 
EVA crewmember could impart into the feed-through 
fitting with the EVA tool. The SRP chairman asked the 
EVA representative for the EVA crewmember force 
input requirement, and that panel member provided a 
response. Unfortunately, by the time the SRP EVA 
panel representative became engaged in the 
conversation, they’d not followed the entire 

conversation as close as necessary (recall, for over an 
hour the conversation centered around engineering and 
MOD responsibilities) and provided an incorrect input 
requirement. Luckily, after a poll of exiting actions 
required to close out the hazard report in question (a 
function of Debrief for later discussion in this paper), 
the panel member caught the mistake and the safety 
panel provided the hardware provider with the proper 
requirement. 

It was a minor incident, but proves that loss of 
situational awareness is insidious, and difficult with 
potentially tragic consequences. 

3.3 Teamwork Barriers 

This section will address teamwork as it applies to the 
panel setting, in terms of leader/group interface. 

3.4 Decision Making Challenges 

Generally, the decisions made within a panel setting 
come nowhere close to intensity of decisions made in 
the high stress environment of flight operations or 
medical emergency rooms, but this does not relieve 
panels from making effective, timely and definitive 
decisions with respect dispositioning all the various 
reports and analysis brought before the panel. 

Allowing panel members to participate in the decision- 
making process does not mean that all decisions have to 
be made by committee. The degree of participation in 


the decision-making process depends to a considerable 
extent on the organizational culture, and in the context 
of safety review panels, is generally highly reflective of 
the technical competency of the participating panel 
members, the presenting hardware provider’s hazard 
analysis and technical support, and the technical 
perception shared among NASA’s departmental 
agencies. 

Though not overly common, the ability to reach 
unanimous (or even near unanimous) consensus 
becomes impossible to reach sometimes within a 
respectable period of time, if ever. However, prolonging 
a decision in some instances, or the desire to “wait for 
consensus” similarly leads to unexpected negative 
consequences. 

Take for example a set of continuously revisited hazard 
reports related to the ISS Regenerative Environmental 
Control and Life Support System (Regen ECLSS). In 
late 2005, the SRP received a set of very detailed, well 
written hazard reports addressing 13 unique hazards and 
over 100 different causes. Upon completion of that 
review, the SRP closed out many of the hazard reports, 
and over the course of the subsequent 5 months, closed 
all the remaining reports in time for a July 2006 launch 
to the ISS. However, with its impending, activation 
aboard the ISS drawing closer, the operations 
community (MOD) took a closer look and many of the 
initially agreed upon Operational Control and 
Agreement Document (OCAD) controls and rejected a 
good number, subsequently reopening 5 hazard reports. 

Over the ensuing months, MOD held multiple day long 
and three day long meetings with the hardware provider 
who in turn updated the reports per MOD’S direction. 
These reports then returned to the SRP with the 
requested modifications. The SRP expected these 
reports to face minimal confrontation during the panel 
meeting given the amount of pre-coordination these 
reports experienced. Surprisingly though, many of 
these reopened reports met stiff criticism during their 
subsequent review from MOD, thus leaving the reports 
“Approved with Modification”. The hardware provider 
included the modifications, resubmitted the reports, and 
once again, met resistance. This cycle continued two or 
three times, frustrating both MOD and the hardware 
provider to the point where effective and constructive 
communication between the two agencies became 
abrasive at best, and at times quite heated such that now 
these two organizations require an additional mediator 
to preside between discussions with these two factions. 

The irony of this situation is that just about every panel 
member agrees that some of the reopened hazard reports 
in debate actually only constitute marginal hazards at 
best, let alone something requiring months of debate 



and argument between two NASA centers justifying the 
deterioration of inter-agency relationship. 

In instances such as these, though an inability to reach 
“consensus” exists, the ability to discern necessary tasks 
from those deemed not important, and then make a 
solid, technically sound decision and defend that 
position preserves invaluable relationships while 
maintaining, if not enhancing credibility within the 
panel and safety community. In other words, know 
when to say “Enough”, and move on. 

3.5 Debriefing Necessities 

In general, a debrief consists of reviewing and 
discussing safety review accomplishments, recapping 
the achievements of the review, addressing the 
encountered barriers, and discussing how to better 
accomplish the review next time. 

Admittedly, after a long review (much like after a long 
and exhausting flying mission or 18 car ride across the 
mid-west plains states), a serious case of “get home-itis” 
exists, and the last thing anyone wants is to prolong an 
already long, and sometimes quite tedious work day. 
Debriefing under these conditions then becomes 
difficult, but still necessary for the following reason: 
often, the greatest learning occurs upon the completion 
of the review during the recap of the day’s events. 
Thus, to prove the necessity and/or advantages of post 
review debriefs, consider this next example. 

This paper already discussed the scenario surrounding a 
single incidence resulting from the loss of situational 
awareness and the debrief that eventually rectified the 
initial error, but better examples exist. During a recent 
safety review pertaining to an International Partner’s 
(IP) visiting vehicle, the SRP addressed what just about 
every panel member considers “the most challenging 
hazard report reviewed by the SRP, bar none”. 
Unfortunately, the SRP began reviewing this vehicle 
nearly 9 years ago and today only a few panel members 
still remain from the initial set of members. Even more 
challenging, the assigned safety engineers to this vehicle 
“swapped hands” three times with little to know 
continuity during the handoff. While the minutes from 
past reviews exist, understanding a vehicle, all its 
systems and hazard control logic with something so 
complex becomes difficult from a set of minutes 
becomes extremely difficult. In fact, a number of these 
reports complexity are such that the SRP needs to 
review them on a cause by cause basis as opposed to 
looking at the hazard report as a whole. While each 
panel member makes every attempt possible to prepare 
for such reviews, oftentimes hardware containing this 
much history is beyond many panel member’s ability to 
provide much technical input. 


On multiple occasions I heard panel member quote “I 
just don’t have the history with (this hardware) to feel 
that I can provide much input, so I just sit back and 
listen”. Perhaps not the best attitude, but certainly an 
understandable one. While teamwork and pre -briefs 
help, a solid debrief makes most members feel 
comfortable because they at least confirm what they 
understood transpired, and if they disagree on a subject, 
a debrief offers one final opportunity to address a 
concern, issue, or clarification point. 

Again, take the IP visiting vehicle example. With 
respect to its collision with the ISS, there are at least 21 
different causes, and counting, most of which are 
considerably complex with multiple integrated hazards 
and concerns. Very few panel members can claim 
complete understanding of each and every cause, while 
some understand a small fraction of the discussion. 
With an effective debrief (after the end of a 3 day 
discussion on this one report alone), the safety engineer 
along with the chairmen discuss every action and issue 
driving each and every cause to remain open, or the 
supporting rationale discussing why the SRP agreed to 
close a cause with each panel member present. This 
invites discussion at a more basic level, or jogs 
memories of issues “earmarked” for future discussion. 
In the case of the above mentioned review, several panel 
members expressed great thanks for the ability to clarify 
their notes and in a non-threatening manner confirm, 
reaffirm, or just learn what took place once the three 
day’s worth of material was compressed into a final 15 
minute discussion. 



